Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reencode the media as webm hevc and av1 for smaller files #2491

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from
Draft

Reencode the media as webm hevc and av1 for smaller files #2491

wants to merge 4 commits into from

Conversation

MTRNord
Copy link
Collaborator

@MTRNord MTRNord commented Sep 13, 2024 •
edited
Loading

This is a WIP with the goal to possibly not need the original media anymore but only the smaller versions.

Example result can be seen at https://d020c70a.matrix-website.pages.dev/blog/2024/07/12/this-week-in-matrix-2024-07-12/#voip-call-membership-reliability

It also uses figureVideo consistently.

The reason for both h256 and av1 is that safari only supports av1 when hardware decoding is available. However it has as one of the few great support for h256. So we provide this as a fallback. As the last resort we then currently fall back to the original. However instead it might make sense to provide h264 I guess.

TODOs:

Fix this issue:

Converting sENlWDyoPRAbNtjoqKFhzTTt
ffmpeg version 6.1.2 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 14 (GCC)
  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Wno-complain-wrong-lang -Werror=format-security -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -march=x86-64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,pack-relative-relocs -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -Wl,--build-id=sha1 ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --enable-chromaprint --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-lcms2 --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libbs2b --enable-libcodec2 --enable-libcdio --enable-libdrm --enable-libjack --enable-libjxl --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libharfbuzz --enable-libilbc --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-libplacebo --enable-librsvg --enable-librav1e --enable-librubberband --enable-libsmbclient --enable-version3 --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libshaderc --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-avfilter --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libvpl --enable-runtime-cpudetect
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'sENlWDyoPRAbNtjoqKFhzTTt.mp4':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2022-07-29T11:27:08.000000Z
  Duration: 00:00:12.17, start: 0.000000, bitrate: 3727 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709/bt709/iec61966-2-1, progressive), 1170x2532, 3714 kb/s, 80.02 fps, 600 tbr, 600 tbn (default)
    Metadata:
      creation_time   : 2022-07-29T11:27:08.000000Z
      handler_name    : Core Media Video
      vendor_id       : [0][0][0][0]
      encoder         : H.264
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> av1 (libsvtav1))
Press [q] to stop, [?] for help
Svt[info]: -------------------------------------------
Svt[info]: SVT [version]:	SVT-AV1 Encoder Lib v2.1.0
Svt[info]: SVT [build]  :	GCC 14.1.1 20240522 (Red Hat 14.1.1-4)	64 bit
Svt[info]: LIB Build date: May 26 2024 00:00:00
Svt[info]: -------------------------------------------
Svt[error]: Instance 1: The maximum allowed frame rate is 240 fps
Svt[warn]: SVT-AV1 has an integrated mode decision mechanism to handle scene changes and will not insert a key frame at scene changes
[libsvtav1 @ 0x5606a4e1e640] Error setting encoder parameters: bad parameter (0x80001005)
[vost#0:0/libsvtav1 @ 0x5606a4e1e380] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
Error while filtering: Invalid argument
[out#0/webm @ 0x5606a4e0d740] Nothing was written into output file, because at least one of its streams received no packets.

Also the convert script should have a mode where it only converts missing files

@cloudflare-workers-and-pages Cloudflare Workers and Pages
Copy link

cloudflare-workers-and-pages bot commented Sep 13, 2024 •
edited
Loading

Deploying matrix-website with  Cloudflare Pages  Cloudflare Pages

Latest commit: e237eb4
Status: ✅  Deploy successful!
Preview URL: https://aabe1cf9.matrix-website.pages.dev
Branch Preview URL: https://av1-and-hevc-support.matrix-website.pages.dev

View logs

@MTRNord MTRNord force-pushed the av1_and_hevc_support branch 3 times, most recently from e95109a to 7eeb103 Compare September 13, 2024 14:10
@MTRNord
Fix typos
6b0cd61 
Signed-off-by: MTRNord <[email protected]>
@MTRNord
Copy link
Collaborator Author

MTRNord commented Sep 22, 2024

Turns out the correct thing to do according to MDN is to go webm+VP9 with h264 mp4 fallback. So I will update this to do that

@MTRNord
Add vp9 to script
c6eef7b 
Signed-off-by: MTRNord <[email protected]>
@MTRNord
Add VP9
e237eb4 
Signed-off-by: MTRNord <[email protected]>
@MTRNord
Copy link
Collaborator Author

MTRNord commented Sep 26, 2024

TODO: https://developer.mozilla.org/en-US/docs/Web/Media/Formats/codecs_parameter

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@MTRNord